In this notebook, we will introdcue a few R packages that can be handy to manipulate, summarize and plot the data coming from the PinAPL-py pipeline. Importantly, the methodologies presented here can be applied and expanded to any reasonnably sized dataset.
The packges used are loaded using the following set of commands. This assumes the packages have been installed using the install.package command or the Rstudio Tools>Install Packages menu.
library(dplyr) # data wrangling and dataframe manipulation
library(tidyr) # data wrangling and dataframe manipulation
library(reshape2) # conversion between long and wide dataframe formats
library(ineq) # calculating gini coefficient (and others)
library(ggplot2) # verstile plotting package
library(pheatmap) # clustering and plotting pretty heatmaps
In order to best benefit from this practise and use case, one needs to be familiar with R data structure and manipulation, mostly differentating vector, matrix and dataframes. A primer on the different data types (numeric, integer, character, etc) is also useful.
The data used in this workshop corresponds to a GECKO screen in a cancer cell line, looking for enrichment of shRNA in cells treated with a genotoxic drug for 1h. 4 libraries from each Baseline (T0), Treated (T3_T) and Untreated (T3_U) were sequenced. In the first part we will study the count files, in a second section the significance analysis.
Data Import and Formatting
The pipeline places the count data in different folders. While there is a way to have R crawl through mulitple folders and import the data, we will assume here, for convenience, that the counts files have been placed in the same “counts” folder and renamed after the library ID. We will import the non-normalized counts.
FiRst we write a function that takes the file name (and path), import the content into a dataframe, names the columns, and add a column containing the name of the file.
importCounts<-function(x){
f<-read.delim(x,header=FALSE)
colnames(f)<-c("sgRNA","gene", "value")
f$Library<-x
return(f)
}
Now we apply this function to a list of files in the folder.
files <- list.files(path="./counts",pattern=".tsv",full.names = TRUE)
CountSg<-sapply(files,importCounts,simplify=FALSE)
We get a list of dataframes, which we collapse into one giant dataframe to which we remove the rownames, since they are not relevant.
CountSg<-do.call(rbind,CountSg)
rownames(CountSg)<-NULL
We can now edit the dataframe and separate the library fields into subfields using the function tidyr::separate function
CountSg<-separate(CountSg, Library, sep="[./]",into=c("pre1","pre2","folder","Library","ext1","ext2"),remove=TRUE)
and we keep only the column we care about using dplyr::select
CountSg<-select(CountSg,sgRNA,gene,value,Library)
We have now a large dataframe containing all the counts data
Summary Statistics and Normalization
Here we will use mostly functions from the dplyr package. for example to determine the total number of counts in each library
totalCounts<-CountSg %>% group_by(Library) %>% summarize(Total=sum(value))
totalCounts
We can therefore normalized the counts to the total number of counts in each library and multiply by 1E6 to get an RPM (read per million) unit. For this we can use the function dplyr::mutate, which can assign tghe results of an operation to a new column. But before we specigy the grouping variable.
CountSg<-CountSg %>% group_by(Library) %>% mutate(RPM=value*1E6/sum(value))
We can now verify that all the libraries have the same sum of RPM and can be compared
CountSg %>% group_by(Library) %>% summarize(Total=sum(RPM))
plotting cumulative distribution of counts
Let’s now look at the distribution of RPM in each library. For this, we can use the ggplot2 package to plot the cumulative distribution function. Which is similar to the Lorenz plot generated by PinAPL-Py. Let’s start with one library.
let’s start with one library using the dplyr::filter command
C1<-filter(CountSg,Library=="Control_1")
A typical ggplot command starts with the ggplot fucntion, which specifies the dataframe and the variable to use in the “aesthetic” (x, y, colors, sizes, etc, etc). This function is them modified by adding the type of plot desired, here stat_ecdf(). The RPM values are following an exponential distribution (lots of low covered sgRNA), we will therefore plot their log10. In order to also represent the sgRNA that are not covered in the library (RPM=0), we will add a small number to all RPM. Finally, we specific the size of the plot (in inches) in the header.
ggplot(C1,aes(log10(RPM+0.01)))+stat_ecdf()

We can now change a bit the labels and the size of the font to make it prettier
ggplot(C1,aes(log10(RPM+0.01)))+stat_ecdf()+
ylab("Fraction of sgRNA")+
xlab("log10(RPM)")+
theme(text=element_text(size=20))

and now we can plot all libraies on the same plot, by starting from the full datafrane and adding a color variable to the aesthetic, (adn adjusting the)and adjusting panel wiodth to make room for the legend)
ggplot(CountSg,aes(log10(RPM+0.01),col=Library))+stat_ecdf()+
ylab("Fraction of sgRNA")+
xlab("log10(RPM)")+
theme(text=element_text(size=20))

Now, this is too many libraries to visualize on one plot, we shoudl be able to separate them by timepoint and condition. We need to add another field to the dataframe. For this we will use the function dplyr::mutate seen before and perfrom a logical test based on the name of the library using grepl. grepl returns TRUE if the queries string is present in the tested string, FALSE otherwise. The function ifelse will assign the type to the type variable, as a result of this test.
CountSg<-mutate(CountSg,type=ifelse(grepl("T0",Library),"Baseline","T3"))
CountSg<-mutate(CountSg,type=ifelse(grepl("T3_T",Library),"treated_T3",type))
CountSg<-mutate(CountSg,type=ifelse(grepl("T3_U",Library),"untreated_T3",type))
we can verify the new field, by listing the unique columns
CountSg %>% select(Library,type) %>% unique()
and we can now use the fucntion facet_wrap in ggplot to separate each type on a separate panel, increaseing the width (resp heigth) to make room if we display them in a row (resp. column)
ggplot(CountSg,aes(log10(RPM+0.01),col=Library))+stat_ecdf()+
ylab("Fraction of sgRNA")+
xlab("log10(RPM)")+
theme(text=element_text(size=20))+
facet_wrap(~type,ncol=3)

We can clearly see that some selection happened in the treated sample. Let’s calculte the gini coefficient for each library. this is provided by the function ineq::ineq
gini<-CountSg %>% group_by(Library,type) %>% summarize(gini=ineq(RPM,type="Gini"))
gini
Let’s plot the corresponding bar graph using the geom_bar function. Beause the value to plot is directly given by the gini field and that no math or calculations are needed, we need to specific stat=“identity”
ggplot(gini,aes(Library,gini,fill=type))+geom_bar(stat="identity")+
ylab("gini")+
xlab("Libraries")+
theme(text=element_text(size=14))

Now this is not pretty, we can rotate the x axis labels in the theme variable. But before we need to change the levels of the Name and type variable so that they are not plotted in alphabetical order.
gini$Library<-factor(gini$Library,levels=c("I1R1T0_1","I1R2T0_1","I2R1T0_1","I2R2T0_1","I1R1T3_U","I1R2T3_U","I2R1T3_U","I2R2T3_U","I1R1T3_T","I1R2T3_T","I2R1T3_T","I2R2T3_T"))
Now we can replot, rotatign the x axiss
ggplot(gini,aes(Library,gini,fill=type))+geom_bar(stat="identity")+
ylab("gini")+
xlab("Libraries")+
theme(text=element_text(size=14),axis.text.x=element_text(angle=45,hjust=1))

Multidimentional plotting
Principal component
Selection has happened on all treated samples, but it is not clear wether the same sgRNAs are being selected. We can investigate the similarity in RPM profile by calculating the fisrt two principal components. For this we first need to generate a matrix of sgRNA x Library using the function reshape2:dcast
CountSgMat<-dcast(data=CountSg,sgRNA~Library,value.var="RPM")
head(CountSgMat)
We can now convert this dataframe into a matrix, by assinging sgRNA field to rownames.
rownames(CountSgMat)<-CountSgMat$sgRNA
CountSgMat<-select(CountSgMat,-sgRNA)
CountSgMat<-as.matrix(CountSgMat)
now I can calcualte the principal components
pca<-prcomp(CountSgMat)
This is a list of two matrices, one for the standard deviation of each PC (pca\(sdev) and one with the values of each PC (pca\)rotation).
In order to use ggplot to plot the PC, I need to convert into a dataframe and add the name of each library as a new field to the pca
pca<-as.data.frame(pca$rotation)
pca<-mutate(pca,Library=rownames(pca))
I can now plot using ggplot
ggplot(pca,aes(PC1,PC2,col=Library))+geom_point()

Now let’s make it prettier
pca$Library<-factor(pca$Library,levels=c("I1R1T0_1","I1R2T0_1","I2R1T0_1","I2R2T0_1","I1R1T3_U","I1R2T3_U","I2R1T3_U","I2R2T3_U","I1R1T3_T","I1R2T3_T","I2R1T3_T","I2R2T3_T"))
ggplot(pca,aes(PC1,PC2,col=Library))+geom_point(size=3, alpha=0.5)+
theme(text=element_text(size=14))

NA
We clearly see that all Baseline are similar, all untreated are similar, but each treated is very different, with a very strong outlier. Can I use custome colors and add some labels?
ggplot(pca,aes(PC1,PC2,col=Library))+geom_point(size=3, alpha=0.5)+
theme(text=element_text(size=14))+
scale_color_manual(values=c(rep("grey",4),rep("steelblue",4),rep("tomato",4)))

NA
clustering and heatmap
We can use the same XX
LS0tCnRpdGxlOiAiQ1JJU1BSIHNjcmVlbmluZyAtIFIgcHJhY3RpY2UiCm91dHB1dDogaHRtbF9ub3RlYm9vawotLS0KCkluIHRoaXMgbm90ZWJvb2ssIHdlIHdpbGwgaW50cm9kY3VlIGEgZmV3IFIgcGFja2FnZXMgdGhhdCBjYW4gYmUgaGFuZHkgdG8gbWFuaXB1bGF0ZSwgc3VtbWFyaXplIGFuZCBwbG90IHRoZSBkYXRhIGNvbWluZyBmcm9tIHRoZSBQaW5BUEwtcHkgcGlwZWxpbmUuIEltcG9ydGFudGx5LCB0aGUgbWV0aG9kb2xvZ2llcyBwcmVzZW50ZWQgaGVyZSBjYW4gYmUgYXBwbGllZCBhbmQgZXhwYW5kZWQgdG8gYW55IHJlYXNvbm5hYmx5IHNpemVkIGRhdGFzZXQuIAoKVGhlIHBhY2tnZXMgdXNlZCBhcmUgbG9hZGVkIHVzaW5nIHRoZSBmb2xsb3dpbmcgc2V0IG9mIGNvbW1hbmRzLiBUaGlzIGFzc3VtZXMgdGhlIHBhY2thZ2VzIGhhdmUgYmVlbiBpbnN0YWxsZWQgdXNpbmcgdGhlIGluc3RhbGwucGFja2FnZSBjb21tYW5kIG9yIHRoZSBSc3R1ZGlvIFRvb2xzPkluc3RhbGwgUGFja2FnZXMgbWVudS4gCgpgYGB7cn0KbGlicmFyeShkcGx5cikgIyBkYXRhIHdyYW5nbGluZyBhbmQgZGF0YWZyYW1lIG1hbmlwdWxhdGlvbgpsaWJyYXJ5KHRpZHlyKSAjIGRhdGEgd3JhbmdsaW5nIGFuZCBkYXRhZnJhbWUgbWFuaXB1bGF0aW9uCmxpYnJhcnkocmVzaGFwZTIpICMgY29udmVyc2lvbiBiZXR3ZWVuIGxvbmcgYW5kIHdpZGUgZGF0YWZyYW1lIGZvcm1hdHMKbGlicmFyeShpbmVxKSAjIGNhbGN1bGF0aW5nIGdpbmkgY29lZmZpY2llbnQgKGFuZCBvdGhlcnMpCmxpYnJhcnkoZ2dwbG90MikgIyB2ZXJzdGlsZSBwbG90dGluZyBwYWNrYWdlCmxpYnJhcnkocGhlYXRtYXApICMgY2x1c3RlcmluZyBhbmQgcGxvdHRpbmcgcHJldHR5IGhlYXRtYXBzCmBgYAoKSW4gb3JkZXIgdG8gYmVzdCBiZW5lZml0IGZyb20gdGhpcyBwcmFjdGlzZSBhbmQgdXNlIGNhc2UsIG9uZSBuZWVkcyB0byBiZSBmYW1pbGlhciB3aXRoIFIgZGF0YSBzdHJ1Y3R1cmUgYW5kIG1hbmlwdWxhdGlvbiwgbW9zdGx5IGRpZmZlcmVudGF0aW5nIHZlY3RvciwgbWF0cml4IGFuZCBkYXRhZnJhbWVzLiBBIHByaW1lciBvbiB0aGUgZGlmZmVyZW50IGRhdGEgdHlwZXMgKG51bWVyaWMsIGludGVnZXIsIGNoYXJhY3RlciwgZXRjKSBpcyBhbHNvIHVzZWZ1bC4gCgpUaGUgZGF0YSB1c2VkIGluIHRoaXMgd29ya3Nob3AgY29ycmVzcG9uZHMgdG8gYSBHRUNLTyBzY3JlZW4gaW4gYSBjYW5jZXIgY2VsbCBsaW5lLCBsb29raW5nIGZvciBlbnJpY2htZW50IG9mIHNoUk5BIGluIGNlbGxzIHRyZWF0ZWQgd2l0aCBhIGdlbm90b3hpYyBkcnVnICBmb3IgMWguIDQgbGlicmFyaWVzIGZyb20gZWFjaCBCYXNlbGluZSAoVDApLCBUcmVhdGVkIChUM19UKSBhbmQgVW50cmVhdGVkIChUM19VKSB3ZXJlIHNlcXVlbmNlZC4gSW4gdGhlIGZpcnN0IHBhcnQgd2Ugd2lsbCBzdHVkeSB0aGUgY291bnQgZmlsZXMsIGluIGEgc2Vjb25kIHNlY3Rpb24gdGhlIHNpZ25pZmljYW5jZSBhbmFseXNpcy4gCgojIERhdGEgSW1wb3J0IGFuZCBGb3JtYXR0aW5nCgpUaGUgcGlwZWxpbmUgcGxhY2VzIHRoZSBjb3VudCBkYXRhIGluIGRpZmZlcmVudCBmb2xkZXJzLiBXaGlsZSB0aGVyZSBpcyBhIHdheSB0byBoYXZlIFIgY3Jhd2wgdGhyb3VnaCBtdWxpdHBsZSBmb2xkZXJzIGFuZCBpbXBvcnQgdGhlIGRhdGEsIHdlIHdpbGwgYXNzdW1lIGhlcmUsIGZvciBjb252ZW5pZW5jZSwgdGhhdCB0aGUgY291bnRzIGZpbGVzIGhhdmUgYmVlbiBwbGFjZWQgaW4gdGhlIHNhbWUgImNvdW50cyIgZm9sZGVyIGFuZCByZW5hbWVkIGFmdGVyIHRoZSBsaWJyYXJ5IElELiBXZSB3aWxsIGltcG9ydCB0aGUgbm9uLW5vcm1hbGl6ZWQgY291bnRzLiAKCkZpUnN0IHdlIHdyaXRlIGEgZnVuY3Rpb24gdGhhdCB0YWtlcyB0aGUgZmlsZSBuYW1lIChhbmQgcGF0aCksIGltcG9ydCB0aGUgY29udGVudCBpbnRvIGEgZGF0YWZyYW1lLCBuYW1lcyB0aGUgY29sdW1ucywgYW5kIGFkZCBhIGNvbHVtbiBjb250YWluaW5nIHRoZSBuYW1lIG9mIHRoZSBmaWxlLiAKCmBgYHtyfQppbXBvcnRDb3VudHM8LWZ1bmN0aW9uKHgpewogIGY8LXJlYWQuZGVsaW0oeCxoZWFkZXI9RkFMU0UpCiAgY29sbmFtZXMoZik8LWMoInNnUk5BIiwiZ2VuZSIsICJ2YWx1ZSIpCiAgZiRMaWJyYXJ5PC14CiAgcmV0dXJuKGYpCn0KYGBgCk5vdyB3ZSBhcHBseSB0aGlzIGZ1bmN0aW9uIHRvIGEgbGlzdCBvZiBmaWxlcyBpbiB0aGUgZm9sZGVyLgoKYGBge3J9CmZpbGVzIDwtIGxpc3QuZmlsZXMocGF0aD0iLi9jb3VudHMiLHBhdHRlcm49Ii50c3YiLGZ1bGwubmFtZXMgPSBUUlVFKQpDb3VudFNnPC1zYXBwbHkoZmlsZXMsaW1wb3J0Q291bnRzLHNpbXBsaWZ5PUZBTFNFKQoKYGBgCldlIGdldCBhIGxpc3Qgb2YgZGF0YWZyYW1lcywgd2hpY2ggd2UgY29sbGFwc2UgaW50byBvbmUgZ2lhbnQgZGF0YWZyYW1lIHRvIHdoaWNoIHdlIHJlbW92ZSB0aGUgcm93bmFtZXMsIHNpbmNlIHRoZXkgYXJlIG5vdCByZWxldmFudC4KIAogCmBgYHtyfQpDb3VudFNnPC1kby5jYWxsKHJiaW5kLENvdW50U2cpCnJvd25hbWVzKENvdW50U2cpPC1OVUxMCmBgYAoKV2UgY2FuIG5vdyBlZGl0IHRoZSBkYXRhZnJhbWUgYW5kIHNlcGFyYXRlIHRoZSBsaWJyYXJ5IGZpZWxkcyBpbnRvIHN1YmZpZWxkcyB1c2luZyB0aGUgZnVuY3Rpb24gdGlkeXI6OnNlcGFyYXRlIGZ1bmN0aW9uCgoKYGBge3J9CkNvdW50U2c8LXNlcGFyYXRlKENvdW50U2csIExpYnJhcnksIHNlcD0iWy4vXSIsaW50bz1jKCJwcmUxIiwicHJlMiIsImZvbGRlciIsIkxpYnJhcnkiLCJleHQxIiwiZXh0MiIpLHJlbW92ZT1UUlVFKQoKYGBgCmFuZCB3ZSBrZWVwIG9ubHkgdGhlIGNvbHVtbiB3ZSBjYXJlIGFib3V0IHVzaW5nIGRwbHlyOjpzZWxlY3QKYGBge3J9CkNvdW50U2c8LXNlbGVjdChDb3VudFNnLHNnUk5BLGdlbmUsdmFsdWUsTGlicmFyeSkKYGBgCgpXZSBoYXZlIG5vdyBhIGxhcmdlIGRhdGFmcmFtZSBjb250YWluaW5nIGFsbCB0aGUgY291bnRzIGRhdGEKCiMgU3VtbWFyeSBTdGF0aXN0aWNzIGFuZCBOb3JtYWxpemF0aW9uCgpIZXJlIHdlIHdpbGwgdXNlIG1vc3RseSBmdW5jdGlvbnMgZnJvbSB0aGUgZHBseXIgcGFja2FnZS4gZm9yIGV4YW1wbGUgdG8gZGV0ZXJtaW5lIHRoZSB0b3RhbCBudW1iZXIgb2YgY291bnRzIGluIGVhY2ggbGlicmFyeQoKYGBge3J9CnRvdGFsQ291bnRzPC1Db3VudFNnICU+JSBncm91cF9ieShMaWJyYXJ5KSAlPiUgc3VtbWFyaXplKFRvdGFsPXN1bSh2YWx1ZSkpCnRvdGFsQ291bnRzCmBgYAoKV2UgY2FuIHRoZXJlZm9yZSBub3JtYWxpemVkIHRoZSBjb3VudHMgdG8gdGhlIHRvdGFsIG51bWJlciBvZiBjb3VudHMgaW4gZWFjaCBsaWJyYXJ5IGFuZCBtdWx0aXBseSBieSAxRTYgdG8gZ2V0IGFuIFJQTSAocmVhZCBwZXIgbWlsbGlvbikgdW5pdC4gRm9yIHRoaXMgd2UgY2FuIHVzZSB0aGUgZnVuY3Rpb24gZHBseXI6Om11dGF0ZSwgd2hpY2ggY2FuIGFzc2lnbiB0Z2hlIHJlc3VsdHMgb2YgYW4gb3BlcmF0aW9uIHRvIGEgbmV3IGNvbHVtbi4gQnV0IGJlZm9yZSB3ZSBzcGVjaWd5IHRoZSBncm91cGluZyB2YXJpYWJsZS4gCgoKYGBge3J9CkNvdW50U2c8LUNvdW50U2cgJT4lIGdyb3VwX2J5KExpYnJhcnkpICU+JSBtdXRhdGUoUlBNPXZhbHVlKjFFNi9zdW0odmFsdWUpKQpgYGAKCldlIGNhbiBub3cgdmVyaWZ5IHRoYXQgYWxsIHRoZSBsaWJyYXJpZXMgaGF2ZSB0aGUgc2FtZSBzdW0gb2YgUlBNIGFuZCBjYW4gYmUgY29tcGFyZWQKCmBgYHtyfQpDb3VudFNnICU+JSBncm91cF9ieShMaWJyYXJ5KSAlPiUgc3VtbWFyaXplKFRvdGFsPXN1bShSUE0pKQpgYGAKCgojIHBsb3R0aW5nIGN1bXVsYXRpdmUgZGlzdHJpYnV0aW9uIG9mIGNvdW50cwoKTGV0J3Mgbm93IGxvb2sgYXQgdGhlIGRpc3RyaWJ1dGlvbiBvZiBSUE0gaW4gZWFjaCBsaWJyYXJ5LiBGb3IgdGhpcywgd2UgY2FuIHVzZSB0aGUgZ2dwbG90MiBwYWNrYWdlIHRvIHBsb3QgdGhlIGN1bXVsYXRpdmUgZGlzdHJpYnV0aW9uIGZ1bmN0aW9uLiBXaGljaCBpcyBzaW1pbGFyIHRvIHRoZSBMb3JlbnogcGxvdCBnZW5lcmF0ZWQgYnkgUGluQVBMLVB5LiBMZXQncyBzdGFydCB3aXRoIG9uZSBsaWJyYXJ5LgoKCmxldCdzIHN0YXJ0IHdpdGggb25lIGxpYnJhcnkgdXNpbmcgdGhlIGRwbHlyOjpmaWx0ZXIgY29tbWFuZAoKYGBge3J9CkMxPC1maWx0ZXIoQ291bnRTZyxMaWJyYXJ5PT0iQ29udHJvbF8xIikKYGBgCgpBIHR5cGljYWwgZ2dwbG90IGNvbW1hbmQgc3RhcnRzIHdpdGggdGhlIGBnZ3Bsb3RgIGZ1Y250aW9uLCB3aGljaCBzcGVjaWZpZXMgdGhlIGRhdGFmcmFtZSBhbmQgdGhlIHZhcmlhYmxlIHRvIHVzZSBpbiB0aGUgImFlc3RoZXRpYyIgKHgsIHksIGNvbG9ycywgc2l6ZXMsIGV0YywgZXRjKS4gVGhpcyBmdW5jdGlvbiBpcyB0aGVtIG1vZGlmaWVkIGJ5IGFkZGluZyB0aGUgdHlwZSBvZiBwbG90IGRlc2lyZWQsIGhlcmUgc3RhdF9lY2RmKCkuIFRoZSBSUE0gdmFsdWVzIGFyZSBmb2xsb3dpbmcgYW4gZXhwb25lbnRpYWwgZGlzdHJpYnV0aW9uIChsb3RzIG9mIGxvdyBjb3ZlcmVkIHNnUk5BKSwgd2Ugd2lsbCB0aGVyZWZvcmUgcGxvdCB0aGVpciBsb2cxMC4gSW4gb3JkZXIgdG8gYWxzbyByZXByZXNlbnQgdGhlIHNnUk5BIHRoYXQgYXJlIG5vdCBjb3ZlcmVkIGluIHRoZSBsaWJyYXJ5IChSUE09MCksIHdlIHdpbGwgYWRkIGEgc21hbGwgbnVtYmVyIHRvIGFsbCBSUE0uIEZpbmFsbHksIHdlIHNwZWNpZmljIHRoZSBzaXplIG9mIHRoZSBwbG90IChpbiBpbmNoZXMpIGluIHRoZSBoZWFkZXIuIAoKYGBge3IgZmlnLndpZHRoPTQsIGZpZy5oZWlnaHQ9NH0KZ2dwbG90KEMxLGFlcyhsb2cxMChSUE0rMC4wMSkpKStzdGF0X2VjZGYoKQpgYGAKCldlIGNhbiBub3cgY2hhbmdlIGEgYml0IHRoZSBsYWJlbHMgYW5kIHRoZSBzaXplIG9mIHRoZSBmb250IHRvIG1ha2UgaXQgcHJldHRpZXIKCmBgYHtyIGZpZy53aWR0aD00LCBmaWcuaGVpZ2h0PTR9CmdncGxvdChDMSxhZXMobG9nMTAoUlBNKzAuMDEpKSkrc3RhdF9lY2RmKCkrCiAgeWxhYigiRnJhY3Rpb24gb2Ygc2dSTkEiKSsKICB4bGFiKCJsb2cxMChSUE0pIikrCiAgdGhlbWUodGV4dD1lbGVtZW50X3RleHQoc2l6ZT0yMCkpCmBgYAogYW5kIG5vdyB3ZSBjYW4gcGxvdCBhbGwgbGlicmFpZXMgb24gdGhlIHNhbWUgcGxvdCwgYnkgc3RhcnRpbmcgZnJvbSB0aGUgZnVsbCBkYXRhZnJhbmUgYW5kIGFkZGluZyBhIGNvbG9yIHZhcmlhYmxlIHRvIHRoZSBhZXN0aGV0aWMsIChhZG4gYWRqdXN0aW5nIHRoZSlhbmQgYWRqdXN0aW5nIHBhbmVsIHdpb2R0aCB0byBtYWtlIHJvb20gZm9yIHRoZSBsZWdlbmQpCiAKYGBge3IgZmlnLndpZHRoPTUsIGZpZy5oZWlnaHQ9NH0KZ2dwbG90KENvdW50U2csYWVzKGxvZzEwKFJQTSswLjAxKSxjb2w9TGlicmFyeSkpK3N0YXRfZWNkZigpKwogIHlsYWIoIkZyYWN0aW9uIG9mIHNnUk5BIikrCiAgeGxhYigibG9nMTAoUlBNKSIpKwogIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MjApKQpgYGAKCk5vdywgdGhpcyBpcyB0b28gbWFueSBsaWJyYXJpZXMgdG8gdmlzdWFsaXplIG9uIG9uZSBwbG90LCB3ZSBzaG91ZGwgYmUgYWJsZSB0byBzZXBhcmF0ZSB0aGVtIGJ5IHRpbWVwb2ludCBhbmQgY29uZGl0aW9uLiBXZSBuZWVkIHRvIGFkZCBhbm90aGVyIGZpZWxkIHRvIHRoZSBkYXRhZnJhbWUuIEZvciB0aGlzIHdlIHdpbGwgdXNlIHRoZSBmdW5jdGlvbiBkcGx5cjo6bXV0YXRlIHNlZW4gYmVmb3JlIGFuZCBwZXJmcm9tIGEgbG9naWNhbCB0ZXN0IGJhc2VkIG9uIHRoZSBuYW1lIG9mIHRoZSBsaWJyYXJ5IHVzaW5nIGdyZXBsLiBncmVwbCByZXR1cm5zIFRSVUUgaWYgdGhlIHF1ZXJpZXMgc3RyaW5nIGlzIHByZXNlbnQgaW4gdGhlIHRlc3RlZCBzdHJpbmcsIEZBTFNFIG90aGVyd2lzZS4gVGhlIGZ1bmN0aW9uIGBpZmVsc2VgIHdpbGwgYXNzaWduIHRoZSB0eXBlIHRvIHRoZSB0eXBlIHZhcmlhYmxlLCBhcyBhIHJlc3VsdCBvZiB0aGlzIHRlc3QuIAoKYGBge3J9CkNvdW50U2c8LW11dGF0ZShDb3VudFNnLHR5cGU9aWZlbHNlKGdyZXBsKCJUMCIsTGlicmFyeSksIkJhc2VsaW5lIiwiVDMiKSkKQ291bnRTZzwtbXV0YXRlKENvdW50U2csdHlwZT1pZmVsc2UoZ3JlcGwoIlQzX1QiLExpYnJhcnkpLCJ0cmVhdGVkX1QzIix0eXBlKSkKQ291bnRTZzwtbXV0YXRlKENvdW50U2csdHlwZT1pZmVsc2UoZ3JlcGwoIlQzX1UiLExpYnJhcnkpLCJ1bnRyZWF0ZWRfVDMiLHR5cGUpKQpgYGAKd2UgY2FuIHZlcmlmeSB0aGUgbmV3IGZpZWxkLCBieSBsaXN0aW5nIHRoZSB1bmlxdWUgY29sdW1ucwpgYGB7cn0KQ291bnRTZyAlPiUgc2VsZWN0KExpYnJhcnksdHlwZSkgJT4lIHVuaXF1ZSgpCmBgYAogYW5kIHdlIGNhbiBub3cgdXNlIHRoZSBmdWNudGlvbiBmYWNldF93cmFwIGluIGdncGxvdCB0byBzZXBhcmF0ZSBlYWNoIHR5cGUgb24gYSBzZXBhcmF0ZSBwYW5lbCwgaW5jcmVhc2VpbmcgdGhlIHdpZHRoIChyZXNwIGhlaWd0aCkgdG8gbWFrZSByb29tIGlmIHdlIGRpc3BsYXkgdGhlbSBpbiBhIHJvdyAocmVzcC4gY29sdW1uKQogCiAKYGBge3IgZmlnLndpZHRoPTgsIGZpZy5oZWlnaHQ9NH0KZ2dwbG90KENvdW50U2csYWVzKGxvZzEwKFJQTSswLjAxKSxjb2w9TGlicmFyeSkpK3N0YXRfZWNkZigpKwogIHlsYWIoIkZyYWN0aW9uIG9mIHNnUk5BIikrCiAgeGxhYigibG9nMTAoUlBNKSIpKwogIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MjApKSsKICBmYWNldF93cmFwKH50eXBlLG5jb2w9MykKYGBgCgpXZSBjYW4gY2xlYXJseSBzZWUgdGhhdCBzb21lIHNlbGVjdGlvbiBoYXBwZW5lZCBpbiB0aGUgdHJlYXRlZCBzYW1wbGUuIExldCdzIGNhbGN1bHRlIHRoZSBnaW5pIGNvZWZmaWNpZW50IGZvciBlYWNoIGxpYnJhcnkuIHRoaXMgaXMgcHJvdmlkZWQgYnkgdGhlIGZ1bmN0aW9uIGluZXE6OmluZXEKCmBgYHtyfQpnaW5pPC1Db3VudFNnICU+JSBncm91cF9ieShMaWJyYXJ5LHR5cGUpICU+JSBzdW1tYXJpemUoZ2luaT1pbmVxKFJQTSx0eXBlPSJHaW5pIikpCmdpbmkKYGBgCgpMZXQncyBwbG90IHRoZSBjb3JyZXNwb25kaW5nIGJhciBncmFwaCB1c2luZyB0aGUgZ2VvbV9iYXIgZnVuY3Rpb24uIEJlYXVzZSB0aGUgdmFsdWUgdG8gcGxvdCBpcyBkaXJlY3RseSBnaXZlbiBieSB0aGUgZ2luaSBmaWVsZCBhbmQgdGhhdCBubyBtYXRoIG9yIGNhbGN1bGF0aW9ucyBhcmUgbmVlZGVkLCB3ZSBuZWVkIHRvIHNwZWNpZmljIHN0YXQ9ImlkZW50aXR5IgoKYGBge3IgZmlnLndpZHRoPTUsIGZpZy5oZWlnaHQ9NH0KZ2dwbG90KGdpbmksYWVzKExpYnJhcnksZ2luaSxmaWxsPXR5cGUpKStnZW9tX2JhcihzdGF0PSJpZGVudGl0eSIpKwogeWxhYigiZ2luaSIpKwogIHhsYWIoIkxpYnJhcmllcyIpKwogIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MTQpKQpgYGAKCk5vdyB0aGlzIGlzIG5vdCBwcmV0dHksIHdlIGNhbiByb3RhdGUgdGhlIHggYXhpcyBsYWJlbHMgaW4gdGhlIHRoZW1lIHZhcmlhYmxlLiBCdXQgYmVmb3JlIHdlIG5lZWQgdG8gY2hhbmdlIHRoZSBsZXZlbHMgb2YgdGhlIE5hbWUgYW5kIHR5cGUgdmFyaWFibGUgc28gdGhhdCB0aGV5IGFyZSBub3QgcGxvdHRlZCBpbiBhbHBoYWJldGljYWwgb3JkZXIuIAoKYGBge3J9CmdpbmkkTGlicmFyeTwtZmFjdG9yKGdpbmkkTGlicmFyeSxsZXZlbHM9YygiSTFSMVQwXzEiLCJJMVIyVDBfMSIsIkkyUjFUMF8xIiwiSTJSMlQwXzEiLCJJMVIxVDNfVSIsIkkxUjJUM19VIiwiSTJSMVQzX1UiLCJJMlIyVDNfVSIsIkkxUjFUM19UIiwiSTFSMlQzX1QiLCJJMlIxVDNfVCIsIkkyUjJUM19UIikpCmBgYApOb3cgd2UgY2FuIHJlcGxvdCwgcm90YXRpZ24gdGhlIHggYXhpc3MKCmBgYHtyIGZpZy53aWR0aD01LCBmaWcuaGVpZ2h0PTR9CmdncGxvdChnaW5pLGFlcyhMaWJyYXJ5LGdpbmksZmlsbD10eXBlKSkrZ2VvbV9iYXIoc3RhdD0iaWRlbnRpdHkiKSsKIHlsYWIoImdpbmkiKSsKICB4bGFiKCJMaWJyYXJpZXMiKSsKICB0aGVtZSh0ZXh0PWVsZW1lbnRfdGV4dChzaXplPTE0KSxheGlzLnRleHQueD1lbGVtZW50X3RleHQoYW5nbGU9NDUsaGp1c3Q9MSkpCmBgYAoKIyBNdWx0aWRpbWVudGlvbmFsIHBsb3R0aW5nCgojIyBQcmluY2lwYWwgY29tcG9uZW50CgpTZWxlY3Rpb24gaGFzIGhhcHBlbmVkIG9uIGFsbCB0cmVhdGVkIHNhbXBsZXMsIGJ1dCBpdCBpcyBub3QgY2xlYXIgd2V0aGVyIHRoZSBzYW1lIHNnUk5BcyBhcmUgYmVpbmcgc2VsZWN0ZWQuIFdlIGNhbiBpbnZlc3RpZ2F0ZSB0aGUgc2ltaWxhcml0eSBpbiBSUE0gcHJvZmlsZSBieSBjYWxjdWxhdGluZyB0aGUgZmlzcnQgdHdvIHByaW5jaXBhbCBjb21wb25lbnRzLiBGb3IgdGhpcyB3ZSBmaXJzdCBuZWVkIHRvIGdlbmVyYXRlIGEgbWF0cml4IG9mIHNnUk5BIHggTGlicmFyeSB1c2luZyB0aGUgZnVuY3Rpb24gcmVzaGFwZTI6ZGNhc3QKCmBgYHtyfQpDb3VudFNnTWF0PC1kY2FzdChkYXRhPUNvdW50U2csc2dSTkF+TGlicmFyeSx2YWx1ZS52YXI9IlJQTSIpCmhlYWQoQ291bnRTZ01hdCkKYGBgCldlIGNhbiBub3cgY29udmVydCB0aGlzIGRhdGFmcmFtZSBpbnRvIGEgbWF0cml4LCBieSBhc3Npbmdpbmcgc2dSTkEgZmllbGQgdG8gcm93bmFtZXMuIApgYGB7cn0Kcm93bmFtZXMoQ291bnRTZ01hdCk8LUNvdW50U2dNYXQkc2dSTkEKQ291bnRTZ01hdDwtc2VsZWN0KENvdW50U2dNYXQsLXNnUk5BKQpDb3VudFNnTWF0PC1hcy5tYXRyaXgoQ291bnRTZ01hdCkKYGBgCgoKCm5vdyBJIGNhbiBjYWxjdWFsdGUgdGhlIHByaW5jaXBhbCBjb21wb25lbnRzCgpgYGB7cn0KcGNhPC1wcmNvbXAoQ291bnRTZ01hdCkKYGBgClRoaXMgaXMgYSBsaXN0IG9mIHR3byBtYXRyaWNlcywgb25lIGZvciB0aGUgc3RhbmRhcmQgZGV2aWF0aW9uIG9mIGVhY2ggUEMgKHBjYSRzZGV2KSBhbmQgb25lIHdpdGggdGhlIHZhbHVlcyBvZiBlYWNoIFBDIChwY2Ekcm90YXRpb24pLiAKCkluIG9yZGVyIHRvIHVzZSBnZ3Bsb3QgdG8gcGxvdCB0aGUgUEMsIEkgbmVlZCB0byBjb252ZXJ0IGludG8gYSBkYXRhZnJhbWUgYW5kIGFkZCB0aGUgbmFtZSBvZiBlYWNoIGxpYnJhcnkgYXMgYSBuZXcgZmllbGQgdG8gdGhlIHBjYQoKYGBge3J9CnBjYTwtYXMuZGF0YS5mcmFtZShwY2Ekcm90YXRpb24pCnBjYTwtbXV0YXRlKHBjYSxMaWJyYXJ5PXJvd25hbWVzKHBjYSkpCmBgYApJIGNhbiBub3cgcGxvdCB1c2luZyBnZ3Bsb3QgCgpgYGB7ciBmaWcud2lkdGg9NSwgZmlnLmhlaWdodD00fQpnZ3Bsb3QocGNhLGFlcyhQQzEsUEMyLGNvbD1MaWJyYXJ5KSkrZ2VvbV9wb2ludCgpCmBgYAoKTm93IGxldCdzIG1ha2UgaXQgcHJldHRpZXIKCmBgYHtyIGZpZy53aWR0aD01LCBmaWcuaGVpZ2h0PTR9CnBjYSRMaWJyYXJ5PC1mYWN0b3IocGNhJExpYnJhcnksbGV2ZWxzPWMoIkkxUjFUMF8xIiwiSTFSMlQwXzEiLCJJMlIxVDBfMSIsIkkyUjJUMF8xIiwiSTFSMVQzX1UiLCJJMVIyVDNfVSIsIkkyUjFUM19VIiwiSTJSMlQzX1UiLCJJMVIxVDNfVCIsIkkxUjJUM19UIiwiSTJSMVQzX1QiLCJJMlIyVDNfVCIpKQoKCmdncGxvdChwY2EsYWVzKFBDMSxQQzIsY29sPUxpYnJhcnkpKStnZW9tX3BvaW50KHNpemU9MywgYWxwaGE9MC41KSsKICB0aGVtZSh0ZXh0PWVsZW1lbnRfdGV4dChzaXplPTE0KSkKICAKYGBgCgpXZSBjbGVhcmx5IHNlZSB0aGF0IGFsbCBCYXNlbGluZSBhcmUgc2ltaWxhciwgYWxsIHVudHJlYXRlZCBhcmUgc2ltaWxhciwgYnV0IGVhY2ggdHJlYXRlZCBpcyB2ZXJ5IGRpZmZlcmVudCwgd2l0aCBhIHZlcnkgc3Ryb25nIG91dGxpZXIuIENhbiBJIHVzZSBjdXN0b21lIGNvbG9ycyBhbmQgYWRkIHNvbWUgbGFiZWxzPyAKCmBgYHtyIGZpZy53aWR0aD01LCBmaWcuaGVpZ2h0PTR9CgpnZ3Bsb3QocGNhLGFlcyhQQzEsUEMyLGNvbD1MaWJyYXJ5KSkrZ2VvbV9wb2ludChzaXplPTMsIGFscGhhPTAuNSkrCiAgdGhlbWUodGV4dD1lbGVtZW50X3RleHQoc2l6ZT0xNCkpKwogIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXM9YyhyZXAoImdyZXkiLDQpLHJlcCgic3RlZWxibHVlIiw0KSxyZXAoInRvbWF0byIsNCkpKQogIApgYGAKCiMjIGNsdXN0ZXJpbmcgYW5kIGhlYXRtYXAKCldlIGNhbiB1c2UgdGhlIHNhbWUgWFgKCgoKIyBPdGhlciByZXNvdXJjZXMKCkkgZW5qb3kgdGhlIGJsb2cgZnJvbSBTdGV2ZW4gVHVybmVyICJHZXR0aW5nIEdlbmV0aWNzIERvbmUiIGh0dHA6Ly93d3cuZ2V0dGluZ2dlbmV0aWNzZG9uZS5jb20sIHdoaWNoIHByb3ZpZGVzIGdyZWF0IGV4YW1wbGUgb2YgUi9iaW9jb25kdWN0b3IgYXBwbGllZCB0byBnZW5vbWljcy4gCgpGb3IgYSBtb3JlIGNsYXNzaWMgdHJhaW5pZ24gaW4gUiBhbmQgYmlvY29uZHVjdG9yLCByZWZlcmVkIHRvIHRoZSBleGNlbGxlbnQgdHV0b3JpYWwgZnJvbSBUaG9tYXMgR2lyayBodHRwOi8vbWFudWFscy5iaW9pbmZvcm1hdGljcy51Y3IuZWR1L2hvbWUvUl9CaW9Db25kTWFudWFsIAoKRmluYWxseSwgdGhlcmUgYXJlIGEgbnVtYmVyIG9mIGNoZWFyc2hlZXQsIHRoYXQgeW91IG1heSB3YW50IHRvIGtlZXAgY2xvc2UgdG8geW91LiAKZHBseXIgJiB0aWR5ciBodHRwczovL3d3dy5yc3R1ZGlvLmNvbS93cC1jb250ZW50L3VwbG9hZHMvMjAxNS8wMi9kYXRhLXdyYW5nbGluZy1jaGVhdHNoZWV0LnBkZgpnZ3Bsb3QgaHR0cHM6Ly93d3cucnN0dWRpby5jb20vd3AtY29udGVudC91cGxvYWRzLzIwMTUvMDMvZ2dwbG90Mi1jaGVhdHNoZWV0LnBkZgpjb2xvcnMgaHR0cHM6Ly93d3cubmNlYXMudWNzYi5lZHUvfmZyYXppZXIvUlNwYXRpYWxHdWlkZXMvY29sb3JQYWxldHRlQ2hlYXRzaGVldC5wZGYKCg==